Coherent Hierarchical Multi-Label Classification Networks

Hierarchical multi-label classification (HMC) is a challenging classification task extending standard multi-label classification problems by imposing a hierarchy constraint on the classes. In this paper, we propose C-HMCNN(h), a novel approach for HMC problems, which, given a network h for the underlying multi-label classification problem, exploits the hierarchy information in order to produce predictions coherent with the constraint and improve performance. We conduct an extensive experimental analysis showing the superior performance of C-HMCNN(h) when compared to state-of-the-art models.Comment: Neural Information Processing Systems 202

Multi-Label Classification Neural Networks with Hard Logical Constraints

Multi-label classification (MC) is a standard machine learning problem in which a data point can be associated with a set of classes. A more challenging scenario is given by hierarchical multi-label classification (HMC) problems, in which every prediction must satisfy a given set of hard constraints expressing subclass relationships between classes. In this paper, we propose C-HMCNN(h), a novel approach for solving HMC problems, which, given a network h for the underlying MC problem, exploits the hierarchy information in order to produce predictions coherent with the constraints and to improve performance. Furthermore, we extend the logic used to express HMC constraints in order to be able to specify more complex relations among the classes and propose a new model CCN(h), which extends C-HMCNN(h) and is again able to satisfy and exploit the constraints to improve performance. We conduct an extensive experimental analysis showing the superior performance of both C-HMCNN(h) and CCN(h) when compared to state-of-the-art models in both the HMC and the general MC setting with hard logical constraints.Comment: arXiv admin note: text overlap with arXiv:2010.1015

Deep Learning with Logical Constraints

In recent years, there has been an increasing interest in exploiting logically specified background knowledge in order to obtain neural models (i) with a better performance, (ii) able to learn from less data, and/or (iii) guaranteed to be compliant with the background knowledge itself, e.g., for safety-critical applications. In this survey, we retrace such works and categorize them based on (i) the logical language that they use to express the background knowledge and (ii) the goals that they achieve.Comment: Survey paper. IJCAI 202

Knowledge Graph Extraction from Videos

Nearly all existing techniques for automated video annotation (or captioning) describe videos using natural language sentences. However, this has several shortcomings: (i) it is very hard to then further use the generated natural language annotations in automated data processing, (ii) generating natural language annotations requires to solve the hard subtask of generating semantically precise and syntactically correct natural language sentences, which is actually unrelated to the task of video annotation, (iii) it is difficult to quantitatively measure performance, as standard metrics (e.g., accuracy and F1-score) are inapplicable, and (iv) annotations are language-specific. In this paper, we propose the new task of knowledge graph extraction from videos, i.e., producing a description in the form of a knowledge graph of the contents of a given video. Since no datasets exist for this task, we also include a method to automatically generate them, starting from datasets where videos are annotated with natural language. We then describe an initial deep-learning model for knowledge graph extraction from videos, and report results on MSVD* and MSR-VTT*, two datasets obtained from MSVD and MSR-VTT using our method.Comment: 10 pages, 4 figure

Adverse childhood experiences and severity levels of inflammation and depression from childhood to young adulthood: a longitudinal cohort study

Adverse childhood experiences (ACEs) are associated with depression and systemic inflammation in adults. However, limited longitudinal research has tested these relationships in children and young people, and it is unclear whether inflammation is an underlying mechanism through which ACEs influence depression. We examined the longitudinal associations of several ACEs across different early-life periods with longitudinal patterns of early-life inflammation and depression in young adulthood and assessed the mediating role of inflammation. The data came from the Avon Longitudinal Study of Parents and Children (N = 3931). ACEs from the prenatal period through to adolescence were operationalised using cumulative scores, single adversities, and dimensions derived through factor analysis. Inflammation (C-reactive protein) was measured on three occasions (9–18 years) and depressive symptoms were ascertained on four occasions (18–23 years). Latent class growth analysis was employed to delineate group-based trajectories of inflammation and depression. The associations between ACEs and the inflammation/depression trajectories were tested using multinomial logistic regression analysis. Most types of ACEs across all early-life periods were associated with elevated depression trajectories, with larger associations for threat-related adversities compared with other ACEs. Bullying victimisation and sexual abuse in late childhood/adolescence were associated with elevated CRP trajectories, while other ACEs were unrelated to inflammation. Inflammation was also unrelated to depression and did not mediate the associations with ACEs. These results suggest that ACEs are consistently associated with depression, whereas the associations of inflammation with ACEs and depression are weak in young people. Interventions targeting inflammation in this population might not offer protection against depression

ROAD-R: The autonomous driving dataset with logical requirements

Neural networks have proven to be very powerful at computer vision tasks. However, they often exhibit unexpected behaviours, violating known requirements expressing background knowledge. This calls for models (i) able to learn from the requirements, and (ii) guaranteed to be compliant with the requirements themselves. Unfortunately, the development of such models is hampered by the lack of datasets equipped with formally specified requirements. In this paper, we introduce the ROad event Awareness Dataset with logical Requirements (ROAD-R), the first publicly available dataset for autonomous driving with requirements expressed as logical constraints. Given ROAD-R, we show that current state-of-the-art models often violate its logical constraints, and that it is possible to exploit them to create models that (i) have a better performance, and (ii) are guaranteed to be compliant with the requirements themselves

Engineering Multi-Agent Systems: State of Affairs and the Road Ahead

The continuous integration of software-intensive systems together with the ever-increasing computing power offer a breeding ground for intelligent agents and multi-agent systems (MAS) more than ever before. Over the past two decades, a wide variety of languages, models, techniques and methodologies have been proposed to engineer agents and MAS. Despite this substantial body of knowledge and expertise, the systematic engineering of large-scale and open MAS still poses many challenges. Researchers and engineers still face fundamental questions regarding theories, architectures, languages, processes, and platforms for designing, implementing, running, maintaining, and evolving MAS. This paper reports on the results of the 6th International Workshop on Engineering Multi-Agent Systems (EMAS 2018, 14th-15th of July, 2018, Stockholm, Sweden), where participants discussed the issues above focusing on the state of affairs and the road ahead for researchers and engineers in this area

Deep learning with hard logical constraints

Deep learning is becoming increasingly ubiquitous and thanks to its successes, it is likely to be applied in almost every aspect of our lives in the next few years. Its success stories however overshadow the dangers that come with its careless application in the real world. Indeed, even if deep learning models report astonishingly high-performance in terms of accuracy (or any other chosen metric), they do not give any guarantees that the model will not have any unintended behaviour when used in practice. This is particularly dangerous in safety-critical applications, where even a single unforeseen mistake can have severe consequences. Further, with each wrong move, human confidence in this technology falters, slowing down its adoption. Thus, it is extremely important to improve the trustworthiness of these models by reducing, if not completely ruling out, all unintended behaviors. In this thesis, I address the problem of how to build deep learning based models able to (i) guarantee the satisfaction of a given set of requirements, which state the correct behaviour of the model, and (ii) learn from the background knowledge specified in the requirements themselves to improve performance. In particular, I focus on (i) deep learning models for multi-label classification problems, and (ii) requirements modelled as hard logical constraints. In order to achieve such a goal, I started by considering multi-label classification problems with hierarchical constraints, and then incrementally increased the expressivity of the constraints. In the first phase of the project, I focused on hierarchical multi-label classification problems, which are multi-label classification problems with hierarchical constraints over the output space of the form $A_1 \to A$ expressing that $A_1$ is a subclass of $A$. For such problems, I developed a novel model, C-HMCNN{$h$}, which, given a network $h$ for the underlying multi-label classification problem, exploits the hierarchy information to produce predictions guaranteed to satisfy the hierarchy constraints and to improve over $h$'s performance. In the second phase of the project, I considered constraints expressed as normal logic rules, i.e., expressions of the form $A_1, \ldots, A_k, \neg A_{k+1}, \ldots, \neg A_n \to A$. This expression imposes that whenever the classes $A_1, \ldots, A_k$ are predicted, while $A_{k+1}, \ldots, A_n$ are not, then the class $A$ should be predicted. For this problem I developed CCN{$h$}, which is an extension of C-HMCNN{$h$}. This model, given a network $h$ for the underlying multi-label classification problem, is able to (i) produce predictions guaranteed to satisfy the constraints, and (ii) exploit the information contained in the constraints to improve performance. Finally, in order to demonstrate the significance of the problem tackled in this thesis, I created the ROad event Awareness Dataset with logical Requirements (ROAD-R), the first publicly available dataset for autonomous driving with requirements modelled as constraints over the output space and expressed as propositional logic formulas. By virtue of creating ROAD-R, I was able to show that the current state-of-the-art models do not learn the requirements from just the data points. My experimental results indicate that more than 90% of their predictions violate the constraints, and that it is possible to exploit the given requirements to create models that (i) have a better performance, and (ii) are guaranteed to be compliant with the given requirements